Secant penalized BFGS: a noise robust quasi-Newton method via penalizing the secant condition

In this paper, we introduce a new variant of the BFGS method designed to perform well when gradient measurements are corrupted by noise. We show that treating secant condition with penalty approach motivated regularized least squares estimation generates parametric family original update at one extreme and not updating inverse Hessian approximation other extreme. Furthermore, find curvature is relaxed as moves towards approximation, disappears entirely where updated. These developments allow us develop refer Secant Penalized (SP-BFGS) allows relax based on amount noise in measurements. SP-BFGS provides means incrementally controlled bias previous which replace overwriting nature an averaging resists destructive effects can cope negative discuss theoretical properties SP-BFGS, including convergence minimizing strongly convex functions presence uniformly bounded Finally, present extensive numerical experiments using over 30 problems from CUTEst test problem set demonstrate superior performance compared both noisy function evaluations.